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Abstract 

Blind deconvolution (BD) arises in many applications. Without assumptions on the signal 
and the filter, BD does not admit a unique solution. In practice, subspace or sparsity assump¬ 
tions have shown the ability to reduce the search space and yield the unique solution. However, 
existing theoretical analysis on uniqueness in BD is rather limited. In an earlier paper, we 
provided the first algebraic sample complexities for BD that hold for almost all bases or frames. 

We showed that for BD of a pair of vectors in C n , with subspace constraints of dimensions v%\ 
and m2, respectively, a sample complexity of n > TO1TO2 is sufficient. This result is suboptimal, 
since the number of degrees of freedom is merely mi + m2 — 1 - We provided analogus results, 
with similar suboptimality, for BD with sparsity or mixed subspace and sparsity constraints. In 
this paper, taking advantage of the recent progress on the information-theoretic limits of unique 
low-rank matrix recovery, we finally bridge this gap, and derive an optimal sample complexity 
result for BD with generic bases or frames. We show that for BD of an arbitrary pair (resp. all 
pairs) of vectors in C™, with sparsity constraints of sparsity levels si and S2, a sample complexity 
of n > Si +S2 (resp. n > 2 (si+S 2 )) is sufficient. We also present analogous results for BD with 
subspace constraints or mixed constraints, with the subspace dimension replacing the sparsity 
level. Last but not least, in all the above scenarios, if the bases or frames follow a probabilistic 
distribution specified in the paper, the recovery is not only unique, but also stable against small 
perturbations in the measurements, under the same sample complexities. 

Index terms — uniqueness, sample complexity, bilinear inverse problem, low-rank matrix 
recovery 

‘This work was supported in part by the National Science Foundation (NSF) under Grants CCF 10-18789 and 
IIS 14-47879. 
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1 Introduction 


Blind deconvolution (BD) is the bilinear inverse problem of recovering the signal and the filter si¬ 
multaneously given the their convolution! or circular convolution. It arises in many applications, 
including blind image deblurring [l], blind channel equalization [2], speech dereverberation |]3j, 
and seismic data analysis (4] . Without further assumptions, BD is an ill-posed problem, and 
does not yield a unique solution. In this paper, we focus on subspace or sparsity assumptions 
on the signal and the filter. These priors, which render BD better-posed by reducing the search 
space, were shown to be effective constraints or regularizers in various applications [5 10]. How¬ 
ever, despite the success in practice, the theoretical results on uniqueness in BD with a subspace 
or sparsity constraint are limited. 

Recently, the “lifting” scheme - recasting bilinear or quadratic inverse problems, such as blind 
deconvolution and phase retrieval, as rank-1 matrix recovery from linear measurements - has 
attracted considerable attention ran- Choudhary and Mitra [12] showed that identifiability in 
BD (or in any bilinear inverse problem) hinges on the set of rank-2 matrices in a certain nullspace. 
In particular, they showed a negative result that the solution to blind deconvolution with a 
canonical sparsity prior, that is, sparsity over the natural basis, is not identifiable |13| . However, 
the authors did not analyze the identifiability of signals that are sparse over other dictionaries. 
Eldar et al. [14 derived tight sufficient conditions for low-rank matrix recovery. However, 
the authors did not exploit any sparsity priors, and the results do not apply to structured 
measurements that arise in blind deconvolution. 

Using the lifting framework, Ahmed et al. [To] , Ling and Strohmer [l5], and Lee et al. mm] 


proposed algorithms to solve BD with with subspace constraints, mixed constraints, and sparsity 


constraints, respectively. Chi 18 solved BD with mixed constraints, where the sparse spikes do 
not necessarily lie on a grid. They all showed successful recovery using convex programming or 
alternating minimization, which implies identifiability and stability. These results are construc¬ 
tive, being demonstrated by establishing performance guarantees of algorithms. However, the 
guarantees are only shown to hold with high probability. The probability of failure is nonzero, 
and decays in a power-law form as the size of the problem increases. 

In earlier papers [19,20 , we addressed the identifiability up to scaling in single channel blind 
deconvolution under subspace or sparsity constraints. We presented the first algebraic sample 
complexities for BD with fully deterministic signal models. In particular, we showed that for 
BD of a pair of vectors in C", with generic subspace constraints of dimensions mi and m 2 , the 
bilinear mapping is injective if n > m\m 2 - However, the number of degrees of freedom in the 


2 














unknown pair of vectors is only m\ + m 2 — 1, hence the above result is suboptimal. Similarly, 
the sample complexities for BD with sparsity or with mixed constraints are n > 2siS 2 and 
n > 2 sim 2 , respectively, where Si and s 2 denote the sparsity levels of the signal and the filter. 
Here the cost for the unknown support is an extra factor of 2. These results suffer from the 
same suboptimality as the results for the subspace constraints, in comparison to the number of 
degrees of freedom of the continuous-valued unknowns. 

In this paper, we finally bridge this gap. We show nearly optimal sufficient conditions for 
identifiability and stability in blind deconvolution that match the number of degrees of freedom 
in the unknowns. Results are given for the cases of subspace constraints, sparsity constraints, 
or mixed constraints, and for complex or real signal and filter. The results of this paper provide 
the first tight sample complexity bounds, without constants or log factors, for unique and stable 
recovery in blind deconvolution. Such tight bounds were not achieved (either for unique or for 
stable recovery) in any of the previous works [lO 15 16,18 . 


The tight sample complexities in the identifiability results apply to almost alQ bases or 
frames. Given a sufficient number of measurements, the conditions for unique recovery are 
violated only on a set of Lebesgue measure zero. In this sense, these results are deterministic, 
requiring no probabilistic assumptions. As an immediate corollary though, if the bases or frames 
are drawn from any probability distribution that is absolutely continuous with respect to the 
Lebesgue measure (e.g., the entries are jointly Gaussian with a non-singular covariance, or i.i.d. 
following a uniform distribution, etc.), then the results in this paper hold: they imply that the 
signal and the filter are identifiable with probability 1, which is better than being identifiable 
with high probability as in previous works (lO 15 16 j!8 . 


The unique recovery results are complemented by matching stability results. If the bases or 
frames follow a distribution specified later in this paper, then under the same sample complex¬ 
ities as in the identifiability results, the recovery is stable with high probability against small 
perturbations in the measurements. In this paper, the probability of failure decays in an expo¬ 
nential form as the size of the problem increases, faster than the power-law decay in previous 
works (I0j[l5j 16,18 . 


One of the main technical tools for the derivation of our results are results on information- 

theoretic limits of low-rank matrix recovery. Inspired by the brilliant work of Riegler et al. |22| on 

such limits for the real matrix case, we extend the results to complex matrix recovery problems. 

The contribution of our extension is two-fold: (i) we provide a simpler proof that gets rid of 

1 Results of similar nature, in that they apply to “almost all” objects of interest, have been derived for FIR 
multichannel deconvolution 
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and for low-rank matrix recovery 
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some unnecessary technicalities; and (ii) we prove a new concentration of measure inequality 
that enables the extension to the complex case. These results may be of independent interest. 

Although all the main results of this paper are stated and proved for ID circular convolution, 
they translate to 2D or higher-dimensional circular convolutions, by replacing the ID discrete 
Fourier transform (DFT) with 2D or higher-dimensional DFT’s. 

The rest of the paper is organized as follows. In Section [2] we formally state the blind 
deconvolution problem and its connection with matrix recovery. In Section [3] we state our 
main results for the identifiability and stability in blind deconvolution of complex signals and of 
real signals. In Section [4] we extend the result for real matrix recovery 22 to complex matrix 
recovery. We prove the main results in Section [5] and conclude the paper in Section [6] 


2 Problem Statement 

2.1 Notations 

We use lower-case letters x, y, z to denote vectors, and upper-case letters D and E to denote 
matrices. We use F to denote the normalized (unitary) discrete Fourier transform (DFT) 
matrix. Unless otherwise stated, all vectors are column vectors. The dimensions of all vectors 
and matrices are made clear in the context. We use superscript letters to denote subvectors 
or submatrices. For example, the scalar represents the jth entry of x. The vector ZA-U) 
represents the jth row of the matrix D. The colon notation is borrowed from MATLAB. The 
transpose and conjugate transpose to a matrix A are denoted by A T and A* , respectively. The 
inner product of two matrices A and M are denoted by (A,M) = trac e(A*M). We use ||-|| 0 to 
denote the £ 0 “norm”, or number of nonzero entries. We use ||-|| 9 to denote the i 2 norm of a 
vector or the spectral norm of a matrix, and ||-|| F to denote the Frobenious norm of a matrix. 
We use 0 to denote entrywise product. Circular convolution is denoted by ®. 

We say a subset VLm of a linear vector space is a cone, if for every M £ VLm an d every a > 0, 
the scaled vector aM € f 1^4 . The real and imaginary parts of a complex vector are denoted by 
Re(x) and Im(x), respectively. If fl x is a subset of C m , then we use Re(D^) = {Re(x) : x £ fix}, 
and Im(D^) = {Im(x) : x £ fix} to denote the real and imaginary parts of fix- The unit ball 
in R m (with respect to the i 2 norm) centered at the origin is denoted by . Then x + RBr™ 
denotes the ball in R m of radius R centered at x. Similarly, the unit ball in C mi xm2 (with respect 
to the Frobenius norm) centered at the origin is denoted by ZI cmi xm 2 . Then M + RB C m, xm 2 
denotes the ball in C miXm2 of radius R centered at M. We use Vcm(R) = J RBcm dx to 
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denote the volume of a ball of radius R in C m . Here, the multiple integral of a real-valued 
function f(x) over fix C C m is defined as the multiple integral of + y/—l y( m + 1:2m )) 

over {y G K 2m : g 

We say a property holds for almost all vectors/matrices (generic vectors/matrices) if the 
property holds for all vectors/matrices but a set of Lebesgue measure zero. 

2.2 Blind Deconvolution 

In this paper, we study the blind deconvolution (BD) problem with the circular convolution 
model. It is the joint recovery of two vectors uq G C ra and vq G C n , namely the signal and the 
filter |^] given their circular convolution z = uq ® Vo, subject to subspace or sparsity constraints. 
The constraint sets flu and fly are subsets of C n . With these definitions, the BD problem is 
written as follows: 


Find (u,v), 
s.t. u®v = z , 

U G flu, v G fly. 

We consider the following scenarios for the constraints: 

1. (Subspace Constraints) The signal u and the filter v reside in lower-dimensional subspaces 
spanned by the columns of D G C nxmi and E G C nxm2 , respectively, with mi,m2 < n. 
The matrices D and E have full column ranks. The signal u = Dx for some x G C mi . The 
filter v = Ey for some y G C™ 2 . 

2. (Sparsity Constraints) The signal u and the filter v are sparse over given dictionaries 
formed by the columns of D G C raxmi and E G C" xm2 , with sparsity level Si and s 2 , 
respectively. Here mi and m 2 don’t have to be smaller than n. The matrices D and E 
are bases or frames that satisfy the spark condition j23j: the spark, namely the smallest 
number of columns that are linearly dependent, of D (resp. E) is greater than 2si (resp. 
2 S 2 ). The signal u = Dx for some x G C mi with ||x||o < Si- The filter v = Ey for some 
y G C m2 with \\y\\ 0 < s 2 - 

3. (Mixed Constraints) The signal u is sparse over a given dictionary D G C” xmi , and the 
filter v resides in a lower-dimensional subspace spanned by the columns of E G C nxm2 , 

2 Due to symmetry, the name “signal” and “filter” can be used interchangeably. 
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with ?77-2 < n. The matrix D satisfies the spark condition, and E has full column rank. The 
signal u = Dx for some x £ C mi with ||x||o < Si- The filter v = Ey for some y £ C m2 (^] 

In all three scenarios, the vectors x, y, and 2 reside in Euclidean spaces (C mi , C m2 and C". 
Given the measurement 2 = {Dx 0 ) © {Eyo), the blind deconvolution problem can be rewritten 
in the following form: 


(BD) Find {x,y), 

s.t. {Dx) © {Ey) = z, 
x £ fix, y £ fly- 

If D and E satisfy the full column rank condition or the spark condition, then the uniqueness 
of {u,v) is equivalent to the uniqueness of {x,y). Indeed, the full rank or spark conditions are 
satisfied for almost all D and E. Therefore, the results about the recovery of {x, y) in BD with 
generic bases or frames imply the corresponding results for (it, v). For simplicity, we will discuss 
problem (BD) from now on. The constraint sets fix and fiy depend on the constraints on the 
signal and the filter. For subspace constraints, fix and fly are C mi and C m2 , respectively. 

For sparsity constraints, fix and fly are {x £ C mi : ||x|| 0 < Si} and {y £ C ™ 2 : ||y||o < s 2 }, 
respectively. 

2.3 Identifiability up to Scaling 

An important question concerning the blind deconvolution problem is to determine when it 
admits a unique solution. The BD problem suffers from scaling ambiguity. For any nonzero 
scalar a £ C such that ax 0 £ fix and ^y 0 £ fly, {D(axo)) © {E{^y 0 )) = {Dx 0 ) © {Ey 0 ) = z. 
Therefore, BD does not yield a unique solution if fix, y contain such scaled versions of xq, yo 
(which is the case for the subspace or sparsity constraint sets in the previous section). Any valid 
definition of unique recovery in BD must address this issue. Our approach is as follows. If every 
solution {x, y) is a scaled version of {x$, yo), then we say that (xo, yo) can be uniquely identified 
up to scaling|^]We also consider the case when this property is satisfied by all pairs {xo,yo) of 
interest. Thus we define identifiability as follows. 

4 We can also consider the scenario where u resides in a subspace spanned by the columns of D, and v is sparse 
over E. By symmetry, the analysis will be almost identical, and thus omitted. 

4 Unconstrained BD suffers also from shift ambiguity. If the signal and the filter are circularly shifted by l and 
—l, respectively, their circular convolution remains the same. However, the BD problem with generic basis or frames 
does not suffer from shift ambiguity. If the signal and the filter are shifted, then they no longer reside in the same 
generic subspaces, or are no longer sparse with respect to the same generic dictionaries, as before. 
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Definition 2.1. 


1. Weak identifiability : We say that the pair {xo,yo) £ fix x fly, in which iq / 0 and 
yo 7 ^ 0, is identifiable up to scaling, if every solution ( x,y) £ fix x fly satisfies x = ax o 
and y = fy 0 for some nonzero a. 

2. Strong identifiability: We say that the set fix * fly is identifiable up to scaling, if every 
pair ( Xo,yo ) £ fix x fly that satisfies Xq ^ 0 and y 0 ^ 0 is identifiable up to scaling. 

For blind deconvolution, there exists a linear operator Gde '■ C miXm2 —>• C ra such that 

GDE{xy T ) = (Dx) © ( Ey ). (1) 

Given the measurement z = GDE(x 0 y (f) = ( Dx 0 ) © ( Ey 0 ), one can recast the BD problem as 
the recovery of the rank-1 matrix Mq = xoyff £ fix = {xy T : x £ flx,y £ fly}. Using this 
so-called “lifting” (To] procedure, the lifted BD problem has the following form: 

(Lifted BD) Find M, 

s.t. Gde(M) = z, 

M £ fix. 


The uniqueness of M 0 is equivalent to the identifiability of (xq, yo) up to scaling. In (Lifted BD), 
weak identifiability means the recovery of Mq is unique, or Mq is the only point in fix that 
maps to Gde(Mq). Strong identifiability means the recovery of all matrices in fix is unique, 
that is Gde is injective on fix. i.e., there exists G~oe '■ Goe^m) f^M- 

Since fix and fly are cones, the lifted constraint set fix is also a cone. As shown later, for 
the linear operator Gde and the cone constraint set fljoh identifiability on fix is essentially the 
same as identifiability on the constraint set restricted to the unit ball fix H'®C m i Xm 2 • From 
now on, we use the shorthand notation 


fits ■■= fix P|#i 


'£m 1 Xm 2 • 


( 2 ) 


Hence a fig = fix D at3c mi xm 2 ■ 
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2.4 Stable Recovery 


Noise is ubiquitous in real-world applications. In a noisy setting, the measurement in matrix 
recovery is z = Gde{Mq) + e, where Mq = Xqi/q denotes the true rank-1 matrix, and e denotes 
noise or other perturbation in the measurement. In order to estimate My from the measurement 
z, we consider the following constrained least squares problem: 

(Noisy BD) min. \\Gde{M) - z\\ 2 , 
s.t. M G crflg, 

where crf2g = {xy T : x G f lx,y G fly, ||x 2 / T || F < cr}. For all practical purposes, the solution 
to a blind deconvolution problem is bounded. Therefore, we solve (Noisy BD) subject to the 
constraint set restricted to a ball, whose radius a is sufficiently large. 

We introduce the following two notions of stability of recovery: 

Definition 2.2. 

1. Single point stability: We say that the recovery of Mq G erflg, using measurement opera¬ 
tor Gde and constraint set erfig, is stable, if for all M G a fig such that || Gde(M) — Gde{Mo)\\ 2 
5, we have \\M — Mo|| 2 < e. 

2. Uniform stability: We say that the recovery on crflg is uniformly stable if for all 
Mi,M 2 G erflg that satisfy \\Gde{Mi) — Gde(M 2 )\\ 2 < 5, we have \\Mi - M 2 \\ 2 < £■ 

In both definitions, e = e(S) is a function of S that vanishes as S approaches 0. 

It is easy to see that the stability as defined above, would guarantee the accuracy of the 
constrained least squares estimation. Let Mi = Xiyf denote the solution to (Noisy BD). 
Suppose the perturbation e is small, i.e., ||e|| 2 < | for some small S > 0. Then the deviation of 
Gde(Mi) from Gde{M 0 ) is small, i.e., 

\\Gde(Mi) — Gde(Mq) || 2 < \\Gde{Mi) — 2:|| 2 +||2 - Gde(M 0 )\\ 2 < 2 \\Gde(M 0 ) - z\\ 2 = 2 ||e|| 2 < S. 

By the definition of single point stability (or uniform stability), we have \\M± — AT 0 11 2 < e(5), 
which is also a small quantity. 

If the recovery of Mq is stable, then for every £ > 0, there exists <5 > 0 such that for every 
M G erfig that satisfies || Gde(M) — Gde(Mo) || 2 < we have \\M — M 0 1| 2 < e. If the recovery is 
uniformly stable on erflg, then for every £ > 0, there exists <5 > 0 such that for all Mi, M 2 G crflg 
that satisfy \\Gde(Mi) — Gde(M 2 )\\ 2 < S, we have \\Mi — M 2 \\ 2 < £. If Gde satisfies strong 



identifiability, i.e., Gde is invertible when restricted to flj\4> then single point stability at Mq 
implies that G~qe is continuous at Gde(M q). Finally uniform stability on erf2g implies that Gjje 
is uniformly continuous on Gde(^^b)- 

Suppose ttju is a cone, and we need to evaluate stability on crffg = flM f) o'£>c m i Xm 2 • We 
can scale Mq and the radius of the ball by ^ simultaneously. If for all M £ fig such that 
\\Gde(M) — Gde{^-) || 2 < 5, we have ||M— ^|| 2 < e(<5), then for all M £ crSlg such that 
II Gde(M) — Gde{Mq)\\ 2 < S , we have \\M — Mq || 2 < cre(^). Therefore, we only need to consider 
the stability of recovery on the constraint set restricted to the unit ball, fig. 

In the next section, we present the main results on the identifiability and stability in blind 
deconvolution, i.e., the optimal sample complexities that guarantee unique and stable recovery 
in (Lifted BD) and (Noisy BD), respectively. 


3 Main Results 


3.1 Identifiability Results 


Subspace membership and sparsity have been used as priors in blind deconvolution for a long 
time. Previous works either use these priors without theoretical justification |5jj9], or impose 
probabilistic models and show successful recovery with high probability |l0 15j 16,18]. The 
sufficient conditions for the identifiability in BD in our prequel paper [T9 are (except for a 
special class of so-called sub-band structured signals or filters) suboptimal. In this section, we 
present sufficient conditions for identifiability in BD, as defined in Section |2.1| with minimal 
assumptions. First, the weak identifiability results in the following theorem are sharp to within 
one sample. 


Theorem 3.1 (Weak Identifiability). Ifn>d, then for almost all D £ C nxmi and E £ C nxm2 , 
the pair ( Xq,pq ) £ fix x fly (xq ^ 0, j/ 0 7 ^ 0 ) is identifiable up to scaling. Here, d is the sample 
complexity bound, which is mi + m 2 , s 1 + m 2 , and si + S 2 in the subspace, mixed, and sparsity 
constraints scenarios, respectively. 


The above sufficient condition is appealing since it approaches the information-theoretic limit 
of blind deconvolution. In BD with subspace, mixed, and sparsity constraints, the number of 
degrees of freedom in the unknowns is mi + m 2 — 1, Si + m 2 — 1, and Si + S 2 — 1, respectively. 
Therefore, to within one sample difference, the sample complexity presented above is optimal. 

This result is a sufficient condition for weak identifiability. Unlike our results on BD with 
generic bases or frames in |[l9], which guarantee the injectivity of the bilinear mapping of circular 
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convolution, this result only guarantees the identifiability of one pair (cco, 2 /o) in the constraint 
set. A sufficient condition for strong identifiability, which applies uniformly to all pairs (£ 0 , 1 / 0 ) 
in the constraint set, is presented next. In comparison to the optimal result in Theorem |3.1| 
the cost for strong identifiability is a factor of 2 in the sample complexity. 

Theorem 3.2 (Strong Identifiability). If n > 2d, then for almost all D £ C nxm ’ and E £ 
C" X7 "2, all pairs ( £o,2/o ) £ fix x fly (xq 7 ^ 0, j/o ^ 0) are identifiable up to scaling. Here, d is 
the same as in Theorem I3.il 


The above results hold true for almost all complex matrices D and E. However, in many 
real-world applications, both the signal and the filter are real vectors. Therefore, it is worthwhile 
to consider the special case where D £ R nxmi , E £ K" xr " 2 , x € R mi , and y £ R m2 . We show 
that the same sample complexities still hold in this special case. 


Theorem 3.3. In the case where D, E, x, and y are real, the sample complexities in Theorems 


3.1 and 3.2 hold for almost all D £ R nxmi and E £ M nxm2 . 


All the results in this section are proved in Section [5] They hold for almost all matrices 
D and E. When the sample complexity is met, the identifiability is violated only on a set of 
Lebesgue measure zero in the space of matrices D and E. Therefore, if D and E are drawn from 
a distribution that is absolutely continuous with respect to the Lebesgue measure (e.g., D and 
E are independent random matrices whose entries are i.i.d. following a Gaussian distribution), 
then the identifiability result holds almost surely. 


3.2 Stability Results 

The previous section gives the sample complexities that guarantee the identifiability in BD. 
Next, we show that the same sample complexity can guarantee stability. Recall that Qde and 
fig are defined in 0 and 0, respectively. Here we only consider single point stability and 
uniform stability on fig, which correspond to Definition |2.2| with a = 1. As argued before, 
stability on fig implies stability on an arbitrary bounded set. 

Theorem 3.4. Assume that D £ C nxmi and E £ (C raxm2 are independent random matrices, 
such that the random vectors {(T , D )^’ : ^*}"_ 1 are i.i.d. following a uniform distribution on 
RB c ™i, and {(FE)(i’ : '>*}”' =1 are i.i.d. following a uniform distribution on RBc™ 2 - 

1. If n > d, then, with probability at least 1 — i) n ~ d {^) n , we have single point stability 

on fig. 
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2. If n > 2d, then, with probability at least 1 
on fig. 


C"\w) n 2d (^) n > we have uniform stability 


Here, d is the same sample complexity bound as in Theorems 3.1 and 3.2 Except for a log factor, 
C' and C" only depend on n, mi, m 2 , s\, and S 2 ■ Define C = 648 TO 1 TO 2 ^1 + 2 In ^ ■ The 
explicit expressions for d, C', and C" in the scenarios of subspace, mixed, or sparsity constraints 
are summarized in Table 0 



d 

a 

C" 

Subspace constraints 

mi + m 2 

c n 

n n ~ d 

(■ iC) n 

ryyftlj 2 d 

Mixed constraints 

si + m 2 

fm i\2 c n 

\ Si ) fl'n — d 

/mil 4 (4 CM 

\ SI / n n-2d 

Sparsity constraints 

si + s 2 

/mi\2/m2\ 2 C n 

V si / V s 2 ) n n ~ d 

(mi l4/m2\ 4 (4C) n 

V Sl / V S2 / n n-2d 


Table 1: A summary of the constants in Theorem 


3.4 


The stability results of Theorem |3.4| correspond to the identifiability results for the complex 
case, in Theorems |3.1| and |3.2| Similar stability results can be derived for the case where D, 
E, x , and y are real, which correspond to the identifiability results in Theorem |3.3| They are 
omitted here for brevity. 

In the discussion below, we interpret the single point stability result in Theorem |3.4| The 
uniform stability result can be interpreted similarly. Here, to make sure that the probability of 
stable recovery 1 — is non-trivial, let e = e(d) > C" 3 ™ (^ 2 )°, where a = 1— ^ € 

(0,1), and e(5) vanishes as 6 approaches 0. 

Reconstruction signal-to-noise ratio (RSNR) and measurement signal-to-noise ratio (MSNR) 
are defined by: 


RSNR = 


\\M-Mowr 


MSNR = 


\\Gde(M 0 )\\1 
II Gde(M) - Qde(M, o)||2 


Consider the case when the error bounds are tight: \\M — Mq\\ 2 = e, and \\Gde{M) — Gde(Mq)\\2 
S. Since the matrix Mo resides in the unit ball, RSNR is on the order of Since {(FD)^’'^*}^ =1 
and {(FE)(E-)*jn =i are uniformly distributed on balls of radius R, the norm of the measure¬ 
ment Gde(M, q) is on the order of R 2 . Hence MSNR is on the order of jt. Theorem 


3.4 


can 


then be interpreted as follows: the probability of failure (unstable reconstruction) is roughly 
RSNR" • MSNR~ ( " -d) . 

Let e(<5) = C /J " (^) 2 > where a = 1 — then the probability of single point stability in 
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Theorem 3.4 reduces to 1 — ( 52 )" d - If 71 > d, then as <5 approaches 0, the recovery error e(8) 


vanishes, and the probability 1 — converges to 1. This means that if D and E are 

random with the distributions specified in Theorem |3.4| then the recovery of M 0 is unique with 
probability 1, which is also a corollary of Theorem |3.1| 

Next, we establish stability for the special case where the operator Gde is an isometry in 
the mean. Given any matrix M = xy T , we have 


G* de Gde(M ) = nY J {FD)^*{FD)^M{FE)^ T {FE)iE\ 

j =1 


the expectation of which is 


E [G* de Gde(M)\ =- 


77117712 

-.2 


-E 


(FD) U ' : 
m\R 2 


2 
2 

m 21? 2 


■E 


(FE) u ’ : 


. 0 * 


M 


m\m.2 mi + 2 m 2 + 2 


M. 


The first line follows from the fact that the distribution of (FD)^’ : ' > * and (FE)^ ,: ^* are indepen¬ 
dent and isotropic. The second line is due to the fact that (FD)^ ,: ^* and (FE)V ,: '* are uniformly 
distributed on RBc m i and RBc m 2 , respectively. It follows that by setting R = ^ (nn+2)(m 2 +2) ^ 4 ^ 
we have E [G* de Gde(M)\ = M. 

Next, as an example, we analyze the uniform stability of the subspace constraints scenario, 
with this special choice of R. This will provide insight into how the constants vary with n, mi, 
and ni 2 - Let s(S) = 2C" 5 " (^■) /3 , where 0=1— _ Substituting the expressions for R 

and C ", and ignoring the log factor, we have s(S) = O |(mim 2 )T^ ni. By Theorem 


3.4 


the subspace constraints scenario, if n > 2 (mi + m 2 ), i.e., 0 £ ( 0 , 1 ), then with probability at 
least 1 — 0.25 n , we have 


1-/3 £ 


\Mi — Mall 2 < (mim.2) 2 n 2 ||^db(Mi) — Gde{M2) 


VMi, M2 £ fig. 


Hence, G0 Fj is Holder continuous of order 0 on Gde(^b)- 

We conclude this section by emphasizing the differences between the identifiability results in 
Section |3.1| and the stability results in Section |3.2| 

1. The identifiability results address the identifiability on cone constraint sets, whereas the 
stability results address the stability on the same constraint sets restricted to a ball of 
an arbitrary but finite radius. From a practical point of view, because the radius can be 
arbitrarily large, this restriction is of no significant consequence. 
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2. The identifiability results hold for generic (Lebesgue almost all) matrices D and E. The 
stability results hold with high probability when D and E follow some specific distributions. 

4 Identifiability in Low-Rank Matrix Recovery 

Using the lifted formulation, blind deconvolution with subspace or sparsity constraints has been 
reduced to the recovery, subject to constraints, of a rank-1 matrix from linear measurements that 
have a particular structure. The identifiability question in BD is thus reduced to identifiabilty in 
the latter recovery problem. In this section we address the more general question of identifiability 
in low rank matrix recovery. Our results express the sample complexity for identifiability in terms 
of the Minkowski dimension of the set in which the matrix to be recovered lives. These results 
are applied to the BD problem in Section [5] to derive the main results of this paper. 

Recently, Riegler et al. (22 derived sample complexity results for low-rank matrix recovery, 
and for the recovery of matrices of low description complexity, that match the number of degrees 
of freedom. They considered the case where the matrices are real. Define the measurement 
operator A : R miXm2 —> R" as 

z = A{M 0 ) = [(Ai, Mo) , {A 2 , Mo), • • • , (. A n , Mo)] T £ R”, 

where the measurement matrices Aj £ R mi xm ' 2 (j = 1,2, ,n). Denoting by C R m i xm2 

the constraint set (which is assumed to be nonempty and bounded) for the unknown matrix, 
the matrix recovery problem is: 


(MR) 


Find 

M, 

s.t. 

A(M) = z. 


M G r2_/V4 • 


The conditions for unique solution to the matrix recovery problem (MR) are expressed in terms 
of the Minkowski dimension of the constraint set flM, which is defined as follows. 

Definition 4.1. The lower and upper Minkowski dimensions of the nonempty bounded set Qm C 

R m 1 xm 2 are 


dim p (Q m ) := liminf 

p—>• 0 


log Nn M ( P) 

lo g p 


dim B (UM) := limsup l0SjV °^ , 
p^o log i 
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where Nq m ( p ) denotes the covering number of LLm given by 


(p) = min< k £ N : Vt M C 


u 

i6{l,2,- 


(Mi + pB s 


! ), Mj £ 


,fc} 


If dim B (r?A^) = dim B (f2jvf), tAen it is simply denoted by dim B (f2j\4). 


The Minkowski dimension of the constraint set Hm can be used to represent its description 
complexity. Riegler et al. showed that the solution to (MR) is unique if the sample complexity is 
greater than the description complexity. For almost all measurement matrices Ai, A 2 , ■ ■ ■ , A n £ 
the recovery of Mq £ f Im is unique if n > dim B (fl_A / f) (see [22| Theorem 1]). An even 
more amazing result is that the same sample complexity can be achieved by rank-1 measurement 
matrices. For almost all a,j £ R mi and bj £ R 1 ™ 2 (j = 1,2, • • • , n), the recovery of Mq £ 
from measurements (cbjbj, M 0 ) = ajM 0 bj (j = 1, 2, • • • , n) is unique if n > dim B (O m) (see 22 
Theorem 2 and Lemma 3]). 

We state and prove the extension of this result to the case where the matrices are complex. 
The Minkowski dimension of the constraint set of complex matrices f 1m C C miX "* 2 can be 


defined as in Definition 4.1 with the real number field R replaced by the complex number field 
C. As will be shown in the next section, by simply changing the number field from real to 
complex, the Minkowski dimension of a set doubles. Meanwhile, by taking n complex-valued 
measurements, the number of real-valued measurements also doubles (from n to 2 n). Theorem 
|4.2| shows that, together with the fact that the Minkowski dimension doubles for the complex 
case, we need the same number of complex-valued measurements in complex matrix recovery as 
we need real-valued measurements in real matrix recovery. 


Theorem 4.2. Suppose the set C C miXm2 is non-empty and bounded. For almost all sets 
of vectors aj £ C 7 ™ 1 and bj £ C m2 (j = 1, 2, • • • ,n), there does not exist a matrix M £ fijvf\{0} 
such that (ajbj, M) = a*Mbj = 0 for j = 1,2, • ■ ■ , n, if 2n > dim B (fl/u). 


Proof. We prove Theorem |4.2| using the following lemma. 

Lemma 4.3. Suppose the set Hm C C miXm2 is non-empty and bounded. Let the vectors 
{aj}™ =1 and {&j}" =1 be independent random vectors, where {aj}" =1 are i.i.d. following a uni¬ 
form distribution on RBc m i, and {&j}" =1 are i.i.d. following a uniform distribution on RBc m 2 ■ 
If 2n > dim B (fl m ), then 


P := P [3M £ f2>i\{0}, s.t. a*Mbj = 0 for j = 1,2, • • • , n] = 0. 
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We use {aj}” =1 , { 6 j-}" =1 ) to denote the event that there exists M £ Ll such that 

a*Mbj = 0 for j = 1,2, • ■ ■ ,n. Restricted to the same support Rt3c™i x Rt3c™2 , the Lebesgue 
measure is absolutely continuous with respect to the uniform distribution. (Because the uniform 
measure is also absolutely continuous with respect to the Lebesgue measure, the two measures 
are equivalent.) If the probability of the event \{0}, {aj}" =1 , { 6 _,}" =1 ) is zero, then the 

Lebesgue measure of the set of {cij }" =1 and {bj}" =1 , over which the event happens, is zero too. 
It follows that, for Lebesgue almost all a,j £ RBc m i and bj £ RBc ™2 (j = 1, 2, • • • , n), the event 
{cij }” =1 , { bj }"_]) does not happen. This argument is true for arbitrary radius R. 


Hence if 2 n > dim P ffl;u), then by Lemma 4.3 the event c/L(H^v(\{0}, {aj}" =1 , {fej}” =1 ) does 
not happen, and therefore this event does not happen for almost all aj £ (C mi and bj £ C ™ 2 
(j = 1,2, ••• , n), i.e., there does not exist a matrix M £ fljvi\{0} such that a*Mbj = 0 for 
j = 1,2,-•• ,n. Therefore, we only need to prove Lemma |4.3[ thus completing the proof of 
Theorem 14.21 □ 


Proof of Lemma \f.3\ The set T2^vj\{0} can be written as 


^.m\{0} = |^J 

Le z+ 

where \,l '■= {M £ LIm : j- < ||M || 2 < L}. By a union bound, we have 

p< E 

Le z+ 

where 

Pl := P [3 M £ s.t. a*Mbj = 0 for j = 1, 2, • • • , n] . 

In order to show that P = 0, it suffices to prove that Pl = 0 for all L £ Z + . 

Let L be an arbitrary positive integer. We form a minimal cover of LLm,l with balls of radius 
p centered at the points {M Pt L,i}i=^ A ’ L ^■ These points may or may not be in LIj^ l- However, 
by the minimality of the cover, the intersection of LIm,l with each ball is nonempty, hence there 
exists another set of points {M' p L such that 

M'p,L,i £ ^M,L + pSc miXr " 2 ), * = R 2, • • ■ ,Nq m L (p). 

Now we cover LIm,l with balls of radius 2 p centered at { M' L which are points in 
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Qm,l (a property that will be needed for inequality Q below), because 


{Mp t L,i + pB C m. 1 X7 n 2 ) C (Mp L^i + 2pi3 C m 1 Xm 2 ), * — 1, 2, • • • ,Nq M ' L (p), 

H M,L C ^ (Mp,L } i + pBfcm iXm 2 ) C U (^, + 2^™). 

l<i<Nn M L (p) l<i<Nn M L (p) 

Defining 5 = R 2 p, we have 


Pl < 

E 

p 

[a m g 

( Kl 

ti + 2 pH C m lXm2 ), s.t. a*jMbj = 0 for j = 1,2, n] 

(3) 


N n M , L ( P ) 






< 

E 

2=1 

p 

[a m g 

(K, l 

+ 2 pB C m 1 xm 2 ), s.t. \a*jMbj < <5 for j = 1 , 2 , n] 

(4) 


N n M , L (p) 






< 

E 

p 

[\a* M ’p 

,L,ibj\ 

< 35 for j = 1, 2, • • • , n] 

(5) 


= E X\v[\a*M' p ^\<Z5} 

i= 1 j=l 

- NnM (w) (M) 2n 9(M,^L,Rr 


( 6 ) 

(7) 


Inequality © uses a union bound. The event in |3]) implies the event in 0 - which then implies 
the event in ([5]). Inequality ([5]) is due to the following chain of inequalities, of which the last is 
implied by a* Mbj | < 5: 

\a*M' p ^\ < \a*{M' pLi - M)V j \ + \a*M^\ 

<||a j || 2 ||M; i)i -M|| 2 || 6 j || 2 + |a*M&;| 

— Il a jll 2 \\M' P! L,i ~ M\\ f \\bj\\ 2 + \a*jMbj\ 

<2 R 2 p + S = 3 5. 


Equation Q is due to the independence between random vector pairs {(ij , bj }” =1 ■ Inequality 
([7]) uses the fact that Nq m L (p) < Nq m (p) = Nq m (^), and the concentration of measure in¬ 
equality P [|a*M( L t bj | < 5] < 5 2 g(5, P, L, R) in Lemma [A.2| in Appendix [ a| (By construction, 
M'p L i: as points in Qm,l, satisfy the norm bounds < L.) Here g(S, jj,L,R) is 


a function of 5 defined in (|A.5[) in Appendix 


, which satisfies lim logg( A A’ g i = q 
s-yo log 7 


Next, we show that 0 implies Pl = 0. Assume the contrary, i.e. Pl > 0. Since Pl 
does not depend on 5 , we have lim inf = 0 - By 0 and the assumed sample complexity 
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2 n > dim B (f}/u), 


0 = liminf^^ 

s^o log | 


< liminf- 
5—>0 


log Nq m (£) + 2n log(3<5) + n logg(3<5, j;,L,R) 

log I 


= dim B (n>i)— 2n < 0, 


which is a contradiction. Since L is arbitrary, we have Pl = 0 for all L £ Z + . This completes 
the proof of Lemma |4.3[ □ 


Corollaries |4.4| and |4.5| are direct consequences of Theorem |4.2| 

Corollary 4.4 (Weak Identifiability, Bounded). Suppose the constraint set Vlj^ C C miXm2 is 
nonempty and bounded. For almost all aj £ C mi and bj £ C m2 (j = 1,2, ••• ,n), the recovery 
of Mq from measurements (ajbj , Mo) (j = 1,2, • ■ • ,n) is unique if 2 n > dim B (Am). 


Proof. Define the set f Im — Mq = {Mi — Mq|Mi £ fix}- Saying that the recovery of Mq from 
a*M 0 bj (j = 1,2, • • ■ , n) is unique, is equivalent to saying that there does not exist a matrix M 
in (£Im ~ iLf 0 )\{0} such that (djbj, M) = 0 (j = 1, 2, • • • , n). 

Since the set f Im — Mq is the shift of the set £Im by Mq, we have that dim B (fl;u — Mo) = 
dim B (Q ka )■ Therefore, Corollary 4.4 follows from Theorem 4.2 □ 


Corollary 4.5 (Strong Identifiability, Bounded). Suppose the constraint set f Im C C miXm2 
is nonempty and bounded. For almost all aj £ C mi and bj £ C m2 (j = 1,2, ••• ,n), the 
recovery of all matrices M 0 £ VLj^ from measurements (ajbj, M 0 ) (j = 1, 2, • • • ,n) is unique if 
n > dime (flaw)- 


Proof. Define the set f Im ~ H m = {Mi — M 2 1Mi, M 2 £ D^vt}. Saying that the recovery of 
all matrices in fljvi is unique, is equivalent to saying that there does not exist a matrix M in 
(f2 m — Ciyvt )\{0} such that (ajbj , M) = 0 (j = 1, 2, • • • , n). 

By Lemma|B.l|in Appendix |B) 


dim B (fl m — D^vi) < dime(flav( — &m) < 2dimB(flav()- 


Therefore, Corollary |4.5| follows from Theorem |4.2| □ 

The proof of Theorem |4.2| is very similar to the proofs of Theorem 2 and Lemma 3 in the 
paper by Riegler et al. (22]. In fact, the proofs in this paper can serve as a simpler proof of the 
sample complexity for the real matrix recovery problem, which is n > dim B (fl m ). by making 
the following modifications: 
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1. Changing the number field from complex to real. 

2. Using a different concentration of measure inequality in |7]): 


| a j M' Pt L ti bj 


<S] <Sf(S,-,L,R), 


which is formally stated and proved in Lemma A.l where f(S, j,L,R) is a function of 
S that satisfies lim = 0. Hence P L < N Qm (£) (3S) n f(3S, j,L, R) n . If 


n > dim p (flA/i), then Pl = 0 for all L £ Z + . 

Owing to the linearity of the measurements in the matrix recovery problem, the above results 
can be easily extended to the case where the constraint set is a cone. To avoid verbosity, we 
only prove Corollary |4.6[ Corollary |4.7| can be proved in a similar fashion. 


Corollary 4.6 (Weak Identifiability, Unbounded). Suppose the constraint set Q.vf C C miXm2 
is a cone. For almost all aj £ C mi and bj £ C m2 (j = 1,2, ••• ,n), the recovery of Mq 
from measurements {o-jbj , Mo) = a*Mobj (j = 1, 2, • • • , n) is unique if 2 n > dim pfilg), where 
Ug = Vt M P| B C m 1 Xm 2 ■ 


Corollary 4.7 (Strong Identifiability, Unbounded). Suppose the constraint set VLm C C miXT ” 2 
is a cone. For almost all aj £ C mi and bj £ C m2 (j = 1,2, ••• ,n), the recovery of all 
matrices M 0 £ from measurements (ajbj ,Mq) = a*M 0 bj (j = 1,2, ,n) is unique if 

n > dimB(Ug), where VLb = H.m P 


Proof of Corollary \4.6\ We prove uniqueness by contradiction. Suppose that the recovery of Mq 
is not unique, i.e., there exists Mi £ VLm such that ( ajbJ,M \) = (aj&J,M 0 ) (j = 1,2, ••• ,n). 
Let a := 2max{||M 0 || F , ||Mi|| f } > 0. Since VLm is a cone, we have 


—M 0 , — Mq £ f!g, 
er a 


ajbj, ~Mi \ = /ajbj, ^Mq 


3 = 1 , 2 , 


Therefore, when the matrix recovery problem is restricted to a nonempty bounded constraint 
set Ug, the recovery of yMo is not unique. This, however, contradicts the sample complexity 

□ 


2 n > dim B (Ug) and Corollary 4.4 


Corollaries |4.6| and |4.7| show that the solution to the matrix recovery problem with a cone 
constraint set is unique, if the solution to the corresponding problem restricted to the unit ball 
is unique. 
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5 Proof of the Main Results 


5.1 Proof of Theorems 13.11 and 13.21 

The identifiability of {xo, yo) up to scaling in (BD) is equivalent to the uniqueness of Mo = XqPo 
in (Lifted BD). Note that 

* = Sde(Mo) = (Dx 0 ) © (Ey 0 ) = \/nF*[(FDxo) © (FEy 0 )\, 

-i( Fz ) w = (FD)^ : ) Xo (FE)U'-) yo = (FD)«’ : ) Xo yT(FE)(F)T = a*M 0 b], 

\Jn 

where aj = (F£))h>0* j s the conjugate transpose of the jth row of FD, and bj = (FE)V ,: >* is 
the conjugate transpose of the jth row of FE. Rewriting (Lifted BD) in the frequency domain: 


(Lifted BD)/ Find M, 

s.t. ctjMbj = -y=(Fz)V\ j = l,2---,n 

M G = {xy T : x G Ll x ,y G fly}- 


Clearly, the constraint set VLm is a cone. Since aj = (FD)^’ : ' > * and bj = (FE)^ ,: ' ) *, there 
exists a bijection between the pair ( D,E ) G C nxm i x C ” xm2 and the set of vector pairs {aj G 

the recovery of Mq is unique for almost all D G C" xmi 


C mi , bj G C m2 }" =1 . By Corollary 


4.6 


and E G C nxm2 if 2n > dim B (fle). By Corollary 4.7 the recovery of all matrices in is 
unique for almost all D G C nxmi and E G C nxm2 if n > dime (fie)- Hence, Theorems 
|3. 2 1 follow from the upper bounds on Minkowski dimensions in Lemma |5.1| 


3.1 


and 


Lemma 5.1. The upper Minkowski dimensions of fig = Ihvi f) ^C m i Xm 2 in (Lifted BD) with 
subspace, mixed, and sparsity constraints are bounded by 2{rn\ +TO 2 ), 2(si + m 2 ), and 2 (si+S 2 ), 
respectively. 


Proof of Lemma \5.1\ For simplicity, we only prove the upper bound for the mixed constraint 
set. The bounds for the other two scenarios can be proved in a similar fashion. First of all, 


He = {xy T ■■ x G fl x ,y G f 1y, ||a-’2/ T || F < 1} 

= {xy T : x G ft x , y G f 1y, ||a;|| 2 < 1, \\y\\ 2 < 1} 
= {xy T : x G fix |^| H<C"*i j V S fly f] #<C m 2 } 
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By Lemmas |B.2| and |B.3[ we have 


dim B (Ox f>c™) <dim B {fix P| B c ™ i) + dim B {fly P B c ™ 2 ) 

<dim B (Re (fl x P Bc™i ) ) + dim B (hn (o* p Bc™i ) ) 

+ dim B (Re (Oy p Bc™2 P + dim B (im (fly P Bc™2 P . (8) 

Recall that, in the mixed constraints scenario, the filter satisfies a subspace constraint, and 
fly = C 7 " 2 . The restriction to the unit ball is fly[^\Bc™2 = ®c m 2 > whose real and imaginary 
parts are B ^ 2 . By a standard volume argument, 

/3 \ m2 

AWp)< (-) • (9) 

Hence 


dim B ^Re (fly P S C m 2 ) ) =dim B ^Im (^Oy P Be " 


=dim B (Z?Rm 2 ) 

!°g N B R m 2 { P ) 


=lim sup 

p—>• 0 


lo Sp 


x- log p 

<lim sup m 2 - y 

p^O " log - 
=m 2 . 


( 10 ) 


Meanwhile, the signal satisfies a sparsity constraint, and fix ={i£ C mi : ||x||o < Si}. The 
restriction to the unit ball is flxC\Bc™i = {x G C mi : ||x||o < Si, ||cc ||2 < 1}, whose real and 
imaginary parts are 

Re(0 A -pH cmi ) =Irn(0^pB cmi ) ={ ie r> : ||x|| 0 < si, ||x|| 2 < 1}, 


which is the union of unit balls in si-dimensional subspaces. Denote this set by T")^. By a 
standard volume argument, 


N r ™-i {p) < 
s 1-1 



Si 


< 


/ era 1 

V s i 


Sl 


Si 
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where the second inequality follows from Stirling’s approximation. Hence, 


dime ^Re (n* n 6cmi )) =dimB ( im ( n * n Bcmi )) 

=dim B (r”i 1 ) 

lOgiNTpmi^p) 

=lim sup-^- 

p^o log - 

log I + log 

<limsup Sl---;- 

p^O log - 

=«!■ ( 11 ) 


Combining ([ 8 |, (10 1 , and 0 , we have that the upper Minkowski dimension of the mixed 


constraint set is bounded by 2 (si + m 2 ). 


□ 


5.2 Proof of Theorem 13.31 


Next, we prove Theorem |3.3[ which establishes results corresponding to those of Theorems |3.1| 
and |3.2| in the case where D, E, x, and y are real. When D are E are real matrices, aj = 
(FD)h-=)* and bj = (. FE)&■> are complex vectors, but they are no longer generic. Therefore, 
Corollaries |4.6| and |4.7| cannot be applied directly to this case. 


Proof of Theorem \3.3\ By © in the proof of Theorem |5.l[ when x, y, and M = xy T are real, 


the Minkowski dimensions of the restricted constraint sets are half those in Theorem 15.11 For 
subspace, mixed, and sparsity constraints, the upper Minkowski dimensions of the restricted 
constraint sets are bounded by mi + m 2 , si + m 2 , and s 1 + S 2 , respectively. To maintain 
the same sample complexities, we need to show a result analogous to Theorem |4.2| in which 
aj = and bj = (FE)^ ,: ' > * , D and E are real matrices, and n > dim B (fl/u) is sufficient. 

Lemma 5.2. Suppose Qm C R m i xm 2 is a nonempty bounded set. Let D £ am ^ g 

r xm2 , a.j = (. FD )0’’ ; )* and bj = (. FE(j = 1,2,- ,n). For almost all D £ and 

E £ R nxm2 , there does not exist a matrix M £ \{0} such that {ajbj , M'j = a*Mbj = 0 for 

j = 1,2, • ,n, if n > dim B (H A1 ). 


The proof of Lemma [5.2| is very similar to that of Theorem |4.2| In fact, the only difference 
is the following: the mapping between the real matrices D,E and the complex vectors {aj}f =1 , 
{bj}j =1 is no longer a bijection. The vectors ai and bi are real vectors. Due to the conjugate 
symmetry of DFT, the vectors aj and a n + 2 -j is a conjugate pairs, i.e. aj = a n + 2 -j- The same 
is true for bj and b n+ 2 ~j- Therefore, (roughly) the first half of the DFT measurements contain 
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all the information of real-valued unknowns. 


There exists a bijection between D,E and the 


vectors , 


sh \r(™+i)/ 2 i 


Due to this subtlety, in the probabilistic argument (analogous to Lemma 4.3) we assume 


s i r(n+i)/2l 
\ a jfj =1 


{M^i +1)/21 


are i.i.d. random vectors drawn from the following distribution: 


• When n is even, ai, an +1 are i.i.d. real random vectors following a uniform distribution 
on and b\, b^+i are i.i.d. real random vectors following a uniform distribution 

on RB^rn 2 . The vectors {Oj} ?_ 2 are i.i.d. complex random vectors following a uniform 
distribution on RBc™i , and {bj}j =2 are i.i.d. complex random vectors following a uniform 
distribution on RBc m 2 ■ 


• When n is odd, ai is a real random vectors following a uniform distribution on RBk™ i, 
and bi is a real random vectors following a uniform distribution on RB«m 2 . The vectors 
l a j}j =2 are i.i.d. complex random vectors following a uniform distribution on RBc™i , and 

n-\-l 

{bj}jl 2 are i.i.d. complex random vectors following a uniform distribution on RBc m 2 - 

The proof of Lemma [d~TT| is changed correspondingly. (As before, we define <5 = pR 2 .) When 
bounding the probability P&, © and ([T]) now become: 

• When n is even, 


(^) H+i 

pl< e n p < 3 *] 

,=i j=i 

(^) (3^/(3<5,^,i?)) 2 ((35) 2 <7(35,-|-,L, J R)) 5 
(^) (3<J)"/(3«,pi,iZ) 2 fl(3<y,i L.iZ)®" 1 . 


• When n is odd, 


Nn M,L 1 P ) IL 5 ^- 

pl< e n p o°;K^i^3«] 

i=l J=1 

(35/(35, J,L,fi))((M) 2 #|L,l?f 
(35)"/(35,i L,i?) 5 (35,-^,L,i?)^. 


Whether n is even or odd, we have Pl = 0{Nq, m (-^y) 5"). By the same argument as in the 
proof of Lemma 4.3 the sample complexity is n > dim p (flA/D. □ 
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5.3 Proof of Theorem 13.41 


In this section, we establish the stability results in blind deconvolution. The measurement in 
(Noisy BD) can be rewritten in the frequency domain: 


2 =(i) 


J_( F z)^ = (FD) (i ’ : >i 0 (f , fi) (i ’ :) t/o + 
\Jn 



{Fe) U) = a*M 0 bj + e U) , 


where Mo = xoyo , a j = {FD)^ ,: )* f bj = (FE)d^*, and e = Fe. Define linear operator 
A(M) by A{M) = [a{Mbi,a 2 Mb 2 , ■ ■ ■ ,a^Mb ^] T . We rewrite (Noisy BD) in the frequency 
domain: 


(Noisy BD)/ min. \\A(M)-z\\ 2 , 

M 

s.t. M £ (tDb, 


where erflg = {xy T : x £ Llx,y £ Qy, ||a;y T || F < a}. 
Note that 


A{M) = -=FG de (M), and ||A(M)|| 2 = -±= \\Q de {M)\\ 2 . 
y/n Y Tl 


The single point stability result in the subspace constraints scenario in Theorem [34] follows from 

s 


Lemma 


5.3 


, with every 6 replaced by -^=. All other cases can be proved using similar lemmas, 


which we omit here for brevity. 


Lemma 5.3. In (Noisy BD)f with subspace constraints, assume that the random vectors {a/}” =1 
are i.i.d. following a uniform distribution on RBc™i, and {6j}" =1 are i.i.d. following a uniform 
distribution on RBc™ 2 - Let the true matrix be M 0 £ fig = fix f) Bem 1 xm 2 = {xy T : x £ 
Be™ i, y € #c m 2 }. If n > m\ + m 2 , then, with probability at least 


1 — 648 niim 2 1 + 2 In 


2 R 2 

1$6 


R 4 


n—m i —m2 


for all M £ fig such that \\A(M) — A(M 0 )|| 2 < 5, we have \\M — M 0 1| 2 < e. 
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Proof of Lemma\5.3\ We need to bound the following probability of stability: 


P s :=P [VM G fig, if || A{M) - A(M 0 )|| 2 < 5, then \\M - M 0 1 | 2 < e] 

=1 - P [3M G fig, s.t. ||.4(M) - A(M 0 )\\ 2 < 5, and \\M - M 0 1 | 2 > e] 
=1 — P [3M G fig — M 0 , s.t. ||M || 2 > e and ||A(M )|| 2 < 5] 

=:1 - P f’ 


where the probability of failure Pf satisfies: 


Pf =P[3M G fig — Mo, s.t. ||M || 2 > e and ||„4(M )|| 2 < 5] 

<P [ 3M G fig — M 0 , s.t. ||M || 2 > £ and \a*jMbj\ < 5,j = 1,2, • • • , n] 


<Nn,(^) (3S) 2n g(3S,£,2,Ry 


< 


/ \ 2mi+2m2 

( ' ( ^2n f ^-Vc ( 21? 

i 1 j V e 2 -^i (R)-VcMR) V 35 


\ 5 

^6\/2 R 2 


2 mi+ 2 m 2 


< 


ip 


R 4 


2 \ 2mi+2m2 


n—m\—m2 


(3^(^( 1 + 2h f))" 

(6^(3^ + 

1 \ n / / 

f 648 mim 2 f l + 21n^- 


( 12 ) 

(13) 

(14) 


Inequality (12 1 follows from ([7]), with the norm bounds e < ||M || 2 < 2. In (131, the bound on 


the covering number of SI 5 = Qm fl -Bc m i Xm 2 = {xy T : x G Bc™i , y E /?c m 2 } follows from (B.l), 


(B.2) in Appendix |B| and from ([9]) using the following steps: 


Nn B ( ^2) < N B cm! (2^2) N Be »* (2^2) 


< N Bsmi 


( 6 


V2y / 2 R? 


N b , 


< 


( 6\/2 R 2 \ mi \ 2 ( ( 6^2 R 2 


Rm2 1 2^2 R 2 

m 2 \ 2 


( 6 V 2 R 2 


2mi+2m.2 


\ 5 


\ 5 


\ 5 


The expression for < 7 ( 35 , f ,2,1?) is given by (A.51 in Appendix |a| Recall that Vc™i {R) denotes 


the volume of a ball of radius R in C mi . Equation (141 follows from the fact that Vc™(R) = 
V^ 2 m ( R ) = —. That completes the proof. □ 
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6 Conclusions 


We studied the identifiability of blind deconvolution problems with subspace or sparsity con¬ 
straints. The sample complexity results in Section [3] are, to within only one sample, optimal. 
Our results are derived with generic bases or frames, which means they are invalid only on a 
set of Lebesgue measure zero. If we assume that the bases or frames are drawn from a dis¬ 
tribution that is absolutely continuous with respect to the Lebesgue measure on the space of 
bases or frames, then the results hold almost surely. Furthermore, if the bases or frames follow 
a distribution specified in this paper, then under the same sample complexities, the recovery is 
not only unique with probability 1, but also stable with high probability against small pertur¬ 
bations in the measurements. These results provide the first tight sample complexity bounds, 
without constants or log factors, for unique or stable recovery in blind deconvolution. They are 
fundamental to the blind deconvolution problem, independent of algorithms. 

A Concentration of Measure 

Lemma A.l. Suppose a G R mi and b G R m2 are independent random vectors, following uniform 
distributions on andRBsjn 2 , respectively. If a matrix M G R miXm2 satisfies I < ||M|| 2 < 

L, then 

P [|a T M&| < p\ < pf{p,I,L,R), 

where f(p,l , L , R) satisfies lim p _>o l ° s ^ p ’ e i L ’ R ^ = 0. 

°s P 

Proof. Suppose the singular value decomposition (SVD) of M is 

M = UY,V t , 

where U G R m i xm i and V G R m2Xm2 are orthogonal matrices, and £ G R m i xm2 satisfies 
£ < E^ 1 ’ 1 ) = ||Af|| 2 < L. Let a := U T a, and b := V T b, then a and b are also independent 
random vectors, following uniform distributions on RB^ m i and RBg^rn 2 , respectively. 
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Therefore, 


p [ 

\a T Mb 

|<p] 

p 

d r E& 

< P 


Irb^ dd fnB Rm2 M 1 (|a T £6| < P 


lRB s m 1 da fj 


RBm, 


d b 


Ifon (R) ■ Vnm, (R) 


da (2: ' 


d5( 2:m 2) <j>{a~b), 


(A.l) 


1-1 


RB„ 


where Vr^i (i?) = j RB ^ da denotes the volume of a ball of radius R in R mi , and 


<p(a, b) = j da (1) J d6 (1) 1 (ja T E6| < p) • 1 (jd (1) | 2 < R 2 


-(2:mi) 


</ daW / db^l |a (1) & (1) + 

“./-fl J-R V ll M ll 2 


1 < _P 


pit 


2p 


< / da (1) min 

/-« \ ||M|| 2 |a (1) | 


4p f 1 + ]n \\M\\ 2 R? 


,2R 


\\ m \\ 2 
U 

i 


4 p f , LR 2 
<-P 1 + ln- 


■ 1 ( J 6 (1) | 2 < # 2 

P 

\\m\\ 2 , 

(A.2) 
(A.3) 

(A.4) 


Substituting (A.41 into (A.l), we obtain 


P [|a T M6| < p] < 


4 p ■ Vr™!" 1 (R.) ■ V R m 2 -l(R) 
(•fc (i?) • 14-2 (i?) 



Define 


f(p,£,L,R) 


4-TWi / Li? 2 


Clearly, limp_> 0 


log f(p,C,L,R) 

i°si 


= 0. 


□ 


Lemma A.2. Suppose a £ C mi and b £ C m2 are independent random vectors, following uniform 
distributions on RBc m i andRBc m 2 , respectively. If a matrix M £ C miXm2 satisfies £ < ||M|| 2 < 
L, then 

P[\a*Mb\<p\ < p 2 g(p,£,L,R), 


where g{p,£, L, R) satisfies linip_>o 1o s9{p^^l,R) _ q 

° g P 


2:m 2 ) 
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Proof. The proof follows steps mostly analogous to those in the proof of Lemma [A.l| by replacing 
the real field by the complex field. Here, we define a := U T a , and b := V*b. It follows that (A.l) 


(A.21 apply, with the real field replaced by the complex field, and the interval of integration 


[— R, R] replaced by the disk in the complex plane RBc< ■ Then (A.31 - (A.4) are replaced by 


6[a,b) < / da^minf- 

Jrb c1 Vi 


np 




, 7 TR Z 


^l + 21nMk^ 


7 T 2 p 2 ( LR? 

<—A- ( 1 + 2 In- 


P 


In a manner analogous to the proof of Lemma |A.1| , it follows that 


0 


Here we use Vc™i ( R) = f RB da to denote the volume of a ball of radius R in C mi . Define 


, „ r (R)-V cm2 - 1 (R) (, LR 2 \ 

g(p,£,L,R). £2 .y cmi ( R yy cm2 ( R) + 2 p )' (A ’ 5) 


Clearly, lim p _> 0 lo z 9 (pRL.R) = q 


□ 


B Useful Lemmas About Minkowski Dimension 


Lemma B.l. Let fix and fly be nonempty bounded subsets of a normed vector space. Then 

dimB(H^ — fly) < diniB(H^) + dimB(H;y). 

Proof. We cover fix and fly with balls of radius p centered at and {;(/,} 

respectively. Given any point x — y £ fix — fly , we can find centers of the above covering, Xi 1 
and j /i 2 , such that 

Wx-x^W <p, \\y - y i2 \\ < p. 


Hence, 


(x-y)- (x h - y i2 )|| < ||a: - ZiJ + ||y - y l2 1| < 2 p. 


Therefore, the set fix — fly can be covered by Nn x (p)Nn y (p) balls of radius 2 p centered at 
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points (like 


yi 2 ) generated by the centers {xi 


\ N n x (p) 
Ji= 1 


and {yij 


Na y (p) 
i=1 


It follows that 


N(n x -n y )(2p) < Nn, x (p)N ny (p). 


We then bound the Minkowski dimension: 


dime (fl * — fly) =limsup 

p—>• 0 

<lim sup 
p—>• o 


logA r (n A -n j; )(2p) 




log N n: , 


2 P 

(p) N Sly(p) 


<lim sup 

p—>-0 


log 


log Afa* (p) 


2p 


lo s^ 

=dim B (f2;t) + dim B (fly). 


lim sup 

p— 


log Nn y (p) 

lo §i 


□ 

Lemma B.2. Let fix and fly be nonempty bounded subsets of C mi and C m2 , respectively. Let 
flM = {xy T '■ x € flx,y € fly} C c mixm2 . T/ien dinr B (fl^vi) < dim B (fl^) + dim B (fly). 

Proof. Since fix and fly are bounded, there exists a large enough constant L such that 

fix C LBc m i , fly C LBc ™2 ■ 

We cover fix and fly with balls of radius p centered at the following two sets of points, respec¬ 
tively: 

{Xi}^ x (P) C LB C , {!/<}& (P) C LB C • 

Given any point xy T £ flM i we can find centers of the above coverings, and fp 2 , such that 


ll®-®iill 2 < p, \\y - y i2 \\ 2 < p. 


Then 


xy T - x. h yl 2 || F = || xy T - x ix y T + x^y 7 - x h yf 2 || p 


— II® ®*i 11 2II2/II2 \\y y *2 II2 ll®*i II2 

<2 Lp. 


Therefore, the set fl_M can be covered by Nq x (p)Nq v (p) balls in C mi XI ™ 2 of radius 2 Lp, centered 
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at the rank-1 matrices (like yf 2 ) generated by the centers of the coverings of fix and fly. It 
follows that 

N QM (2Lp) < N nx (p)N ny (p). (B.l) 


Therefore, 


dime (fix) =limsup 

p—¥ 0 

<lim sup 
p—>• o 


<lim sup 

p—>• o 


log No. M {2Lp) 
lo g2 b 

log Nq x ( p)N ny (p) 
lo S2 h 

log N Qx (p) 


l°g 2pp 
=dim B (n^) + dim B (fly). 


lim S up 1 ° g "°? ( ' ,) 
p ->0 log 2Lp 


□ 

Lemma B.3. Let fix be a nonempty bounded subset of C m . Let Re(fl^) = {Re(x) : x £ fix}, 
and = |Im(a;) : x £ fix}- Then dimB(fl^) < dimB(Re(f2^)) + dimB(Im(fl^)). 

Proof. The real and imaginary parts Re(fl^) and Im(fl^) are bounded subsets of There 
exists a large enough constant L such that 


Re(n*),Im(O0 C LBi 


We cover Re(fl^) and Irn(fl^) with balls of radius p centered at the following two sets of points, 
respectively: 


r x Re \ N ne(n x ) (p) r Imj w im( n x )(p) 

X i Si —1 ’ l i Si—1 


C LB[ 


Given any point x £ fix, we can find centers of the above coverings, a:^ c and x]f\ such that 


Re(a-) - a:^ e || 2 < p, ||lm(x) - x]™\\ 2 < p. 


Let x c = + ^/Tlx 1 ff 1 . Then 

Ik - x c \\ 2 = \J ||R e (rr) - a;^ e || 2 + jjlm(x) - x l ^\\\ < V2p 

Therefore, the set fix can be covered by N Re ^ x ' ) (p)Ni m ^Q x ' ) (p) balls in C m of radius y/2p, 
centered at the complex vectors (like x c ) generated by the centers of the coverings of Re(fl^) 
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and It follows that 


Nn x (V2p) < N Re (Q x )(p)Ni m (Q x )(p). 


(B.2) 


Therefore, 


dim B (fi*) =limsup 1 °^ Vf2 4^^ 

log 7^ 

<lim gup !°g N ne(n x )(p) N im(Q x )(p) 

- T4o P log ^ 

<lim S up l0giVRe ^ )(p) + Hnisup^WP) 

p^° log 7^ log 7^ 

=dim B (Re(f2^)) + dim B (Im(fi^)). 


□ 
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